Graphix-T5: Mixing Pre-trained Transformers with Graph-Aware Layers for Text-to-SQL Parsing

The task of text-to-SQL parsing, which aims at converting natural language questions into executable SQL queries, has garnered increasing attention in recent years. One the major challenges parsing is domain generalization, i.e., how to generalize well unseen databases. Recently, pre-trained text-to-text transformer model, namely T5, though not specialized for achieved state-of-the-art performance on standard benchmarks targeting generalization. In this work, we explore ways further augment T5 model with components parsing. Such are expected introduce structural inductive bias parsers thus improving model’s capacity (potentially multi-hop) reasoning, critical generating structure-rich SQLs. To end, propose a new architecture GRAPHIX-T5, mixed augmented by specially-designed graph-aware layers. Extensive experiments and analysis demonstrate effectiveness GRAPHIX-T5 across four benchmarks: SPIDER, SYN, REALISTIC DK. surpasses all other T5-based significant margin, achieving performance. Notably, GRAPHIX-T5-large reaches superior original T5-large 5.7% exact match (EM) accuracy 6.6% execution (EX). This even outperforms T5-3B 1.2% EM 1.5% EX